TITLE OF THE INVENTION 
IMAGE PROCESSING APPARATUS AND METHOD 



FIELD OF THE INVENTION 

The present invention relates to an image 
processing apparatus and method and, more particularly, 
to an image processing apparatus and method which 
accurately estimate the display color of an acquired 
object, regardless of the conditions of observation 
illuminating light, by using the spectral distribution 
data of the object. 

BACKGROUND OF THE INVENTION 

The method most often used in apparatuses which 
acquire spectral distribution data is to divide into 
bands a wavelength region (to be referred to as a 
"total wavelength region" hereinafter) to be measured 
such as a visible light region, acquire the spectral 
distribution information of each band, and acquire the 
spectral distribution information of the total 
wavelength region from the results. 

Fig. 1 is a block diagram showing the arrangement 
of a multi-spectrum camera 207 for acquiring spectral 
distribution data . 

A plurality of filters having spectral 
characteristics different in wavelength are arranged 
along the periphery of a disk-like rotary filter 202 in 



order of band wavelength. Reflected light from an 
object to be sensed passes through an optical system 
201 and through one filter on the rotary filter 202, 
and is decomposed into light indicative of band 
information. By rotating the rotary filter 202 placed 
before a CCD sensor 204 by a driving motor 203, filters 
facing this CCD sensor 204 are switched, and input 
light is decomposed into light indicating each band 
information . 

The intensity of the light indicating band 
information is converted into a digital signal value by 
the CCD sensor 204 and an A/D converter 205, and stored 
as data in units of pixels into a data storage unit 206. 

Generally, the smaller the number of filters of 
the rotary filter 202, the shorter the processing time 
required for sampling and the higher the speed of image 
sensing. Therefore, the number of filters is minimized. 

As indicated by the solid lines in Fig. 2, the 
spectral distribution data acquired by the 
multi-spectrum camera 207 is the information of each 
band decomposed by the corresponding filter. Hence, 
this data cannot be directly used as the spectral 
distribution data of the total wavelength region. 
Therefore, it is necessary to acquire the spectral 
distribution data of the total wavelength region by 
interpolating the information of each band. Note that 
the spectral distribution data of the total wavelength 



region will be called "multi spectral distribution 
data", with respect to band information acquired by the 
multi-spectrum camera 207. 

In this interpolation, as shown in Fig. 2, an 
intensity value corresponding to the intermediate 
wavelength of each band is defined as band information 
(indicated by symbol • in Fig. 2) . These pieces of 
band information are connected by a curve (indicated by 
the broken line in Fig. 2) to obtain multi spectral 
distribution data as the spectrum interpolation result. 
Fig. 2 shows the spectral characteristic acquired by 
the multi-spectrum camera 207 having six filters. In 
this case, multi spectral distribution data must be 
estimated by using band information having six fixed 
filter characteristics . 

Methods other than the above interpolation 
process are available by which the spectral 
distribution data of the total wavelength region is 
obtained by using band information. One method will be 
explained below. 

A function for obtaining an output value at a 
wavelength X of the spectral distribution data 
information of each decomposed band is defined as 
Ln ( X ) where n = 1, 2, . . . n corresponds to the 
information of each decomposed band. Spectral 
distribution data R ( X ) of the total wavelength region 
is calculated as a linear sum at different wavelengths 



of Ln ( X ) given by 

R(X) = 2 x {al-Ll(X) + a2-L2(l) + + 

an-Ln( X ) } ... (1) 

where al, a2,..., an are arbitrary coefficients which 
can be different from one wavelength X to another 

In addition to the above method, methods 
represented by the KL expanding method are known by 
which the spectral distribution data of the total 
wavelength region is obtained from band information. 
Accordingly, an optimum method is properly used in 
accordance with, e.g., the characteristics of band 
information. 

A technique which realizes perfect color matching 
between environments differing in environmental 
illuminating light, i.e., environments having 
illuminating lights different in spectral distribution 
is disclosed in, e.g., Japanese Patent Laid-Open 
No. 9-172649. Since this technique uses spectral 
distribution data, it is necessary to acquire the multi 
spectral distribution data of illuminating light by 
using the multi-spectrum camera 207 as shown in Fig. 1, 
i.e., to estimate the spectral distribution of 
illuminating light. 

Estimating the spectral distribution requires not 
only the image sensing time of the multi-spectrum 
camera 207 but also, in order to process all band 
information acquired, large amounts of memories for 



storing all band information in units of pixels and 
working buffer memories. In addition, a processing 
time for processing all the band information is 
necessary. 

Furthermore, to avoid errors produced by the 
interpolation process described above, it is necessary 
to use complicated interpolation and increase the 
number of band information. This increases the load 
(time) of image sensing and processing. 

SUMMARY OF THE INVENTION 
The present invention has been made to 
individually or collectively solve the above problems, 
and has as its object to reduce the load (time) of 
image sensing and processing when estimating a spectral 
distribution. 

To achieve the above object, one preferred aspect 
of the present invention discloses an image processing 
apparatus comprising: a first acquisition section, 
arranged to acquire color data of an object; a second 
acquisition section, arranged to acquire a plurality of 
spectral distribution data on the basis of the 
configuration of spectral distribution data defined in 
accordance with the acquired color data; and a 
generator, arranged to generate image data constructed 
by the acquired color data and the plurality of 
spectral distribution data. 



Another preferred aspect of the present invention 
discloses an image processing apparatus comprising: an 
input section, arranged to input image data constructed 
by color data and a plurality of spectral distribution 
data; and an estimator, arranged to estimate spectral 
distribution data of a total wavelength region on the 
basis of the configuration of the spectral distribution 
data defined in accordance with the color data. 

Still another preferred aspect of the present 
invention discloses an image processing apparatus 
comprising: a first acquisition section, arranged to 
acquire color data of an object; a second acquisition 
section, arranged to acquire a plurality of spectral 
distribution data on the basis of the configuration of 
spectral distribution data defined in accordance with 
the acquired color data; an estimator, arranged to 
estimate spectral distribution data of a total 
wavelength region on the basis of the configuration of 
the spectral distribution data; and a generator, 
arranged to generate the spectral distribution data of 
the total wavelength region from the acquired color 
data and the plurality of spectral distribution data, 
on the basis of the estimated spectral distribution 
data of the total wavelength region. 

Other features and advantages of the present 
invention will be apparent from the following 
description taken in conjunction with the accompanying 



drawings, in which like reference characters designate 
the same or similar parts throughout the figures 
thereof . 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram showing the arrangement 

of a multi-spectrum camera for acquiring spectral 

distribution data; 

Fig. 2 is a graph for explaining the spectral 

distribution data acquired by the multi-spectrum 

camera; 

Fig. 3 is a block diagram showing the 
arrangements of an image sensing device for sensing an 
object and an image processing apparatus; 

Fig. 4 is a graph showing the acquisition result 
of eight pieces of band information when a bluish color 
is sensed by the multi-spectrum camera; 

Fig. 5 is a graph showing a method of estimating 
multi spectral distribution data in an embodiment; 

Fig. 6 is a graph for explaining band information 
necessary to estimate multi spectral distribution data 
specifying a skin color; 

Fig. 7 is a view for explaining a look-up table 
stored in a band count value storage; 

Fig. 8 is a view for explaining a look-up table 
stored in a band number storage; 

Fig. 9 is a block diagram showing detailed 



functions of the image sensing device; 

Fig, 10 is a flow chart showing the image sensing 
procedure of the image sensing device; 

Fig. 11 is a block diagram showing detailed 
functions of the image processing apparatus; 

Fig. 12 is a flow chart showing the process 
procedure of the image processing apparatus; 

Fig, 13 is a view showing the relationship 
between multi spectral distribution data and frame band 
information; 

Fig. 14 is a graph for explaining the process of 
compressing band information; 

Fig. 15 is a graph showing the spectral 
characteristic data of RGB; and 

Fig. 16 is a graph showing a color matching 
function in the XYZ colorimetric system. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
An image processing apparatus of an embodiment 
according to the present invention will be described in 
detail below with reference to the accompanying 
drawings . 
[Arrangement] 

Fig. 3 is a block diagram showing the 
arrangements of an image sensing device 101 for sensing 
an image of an object and an image processing apparatus 
104. 



The image sensing device 101 comprises a color 
data sensing unit 102 for acquiring the color data of 
each pixel of an input image, and a multi-spectrum 
camera sensing unit 103 for acquiring that spectral 
distribution data of each pixel of the input image, 
which is defined by the acquired color data. Note that 
the color data sensing unit 102 can also be a color 
data sensing device connected to the multi-spectrum 
camera . 

Note also that the multi-spectrum camera sensing 
unit 103 can be a multi-spectrum image sensing device 
as follows. That is, filters for acquiring band 
spectral distribution data differing in wavelength are 
used in image sensing by the above multi-spectrum 
camera, such that all of these filters are used to 
cover the total wavelength region, thereby always 
acquiring the spectral distribution data of all pixels. 
When this data is stored from the image sensing device 
101 into a data storage unit 105, that spectral 
distribution data of each pixel of the input image, 
which is defined by the acquired color data is stored, 
and the subsequent image processing is performed. 

Furthermore, as will be described in detail later, 
the multi-spectrum camera can also include dedicated 
filters for previously defined specific colors. If it 
is determined that a previously defined specific color 
exists in acquired color data, a dedicated filter for 



the specific color found is used to additionally sense 
all pixels. The multi-spectrum camera sensing unit 103 
can also be a multi-spectrum image sensing device like 
this . 

The arrangement of the multi-spectrum camera 
sensing unit 103 can also be a multi-spectrum image 
sensing device as follows. That is, of filters of the 
multi-spectrum camera, all previously defined filters 
are used to always acquire the spectral distribution 
data of all pixels. When this data is stored from the 
image sensing device 101 into the data storage unit 105, 
that spectral distribution data of each pixel of the 
input image, which is defined by the acquired color 
data is stored, and the subsequent image processing is 
performed. 

The output image signal data (the color data and 
the spectral distribution data defined by this color 
data) from the image sensing device 101 is stored in 
the data storage unit 105 of the image processing 
apparatus 104. 

A color data look-up unit 107 stores information 
for forming main component spectral distribution data 
required by a main component spectral distribution data 
generator 108 to estimate multi spectral distribution 
data by referring to the acquired color data and the 
spectral distribution data. 

A multi spectral distribution data estimator 109 
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estimates the multi spectral distribution data of each 
pixel from the generated main component spectral 
distribution data and/or the spectral distribution data 
defined by the acquired color data. 

The color data look-up unit 107, the main 
component spectral distribution data generator 108, and 
the multi spectral distribution data estimator 109 
construct a multi spectral distribution data estimation 
processor 106. 

A color converter 110 converts the color of the 
multi spectral distribution data of each desired pixel 
by using the estimated multi spectral distribution data. 
A color data converter 111 converts the color-converted, 
multi spectral distribution data of each pixel into 
color data used in an output device 112. 

This output device 112 outputs an image by using 
the output color data from the image processing 
apparatus 104. 

[Multi spectral Distribution Data Estimation Method] 

This embodiment notes that pieces of band 
information necessary to estimate spectral distribution 
data and the number of these pieces of band information 
are specified for a certain arbitrary color. That is, 
the multi-spectrum camera senses spectral distribution 
data in accordance with a color specified by color data 
in an input image. Spectral distribution data is 
estimated by pieces of arbitrary band information 



specified by the color data value of the sensed 
spectral distribution data and the number of these 
pieces of arbitrary band information. In this 
estimation, pieces of band information and the number 
of these pieces of band information are varied in 
accordance with the color data value. 

Also, filters of the multi-spectrum camera are 
prepared for all bands necessary to estimate multi 
spectral distribution data for all color data. For 
example, Fig. 4 is a view showing the result of 
acquisition of eight pieces of band information when a 
bluish color is sensed by the multi-spectrum camera. 
As shown in Fig. 4, if the spectral distribution of 
input light is offset to a specific wavelength region, 
the multi spectral distribution data is estimated by 
interpolating four or five pieces of band information 
of the acquired band information, and the remaining 
four or three pieces of band information are redundant 

In this embodiment, the above offset of the 
spectral distribution is taken into consideration. 
Therefore, if input light is a bluish color, as shown 
in Fig. 5, only three pieces of band information are 
acquired, and the multi spectral distribution data of 
the input light is estimated by interpolation. 
Consequently, as shown in Figs. 4 and 5, it is possibl 
to obtain substantially the same estimation result and 
reduce the number of band information. 
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A method of estimating the multi spectral 
distribution data from band information other than the 
above interpolation method is also available. As 
described in, e.g., "New Version of Color Science 
Handbook 2nd ed. M , page 1196, Fig. 28.29(a), band 
information required to estimate the multi spectral 
distribution data specifying a skin color need only be 
first to third spectral distribution data shown in 
Fig. 6. 

If color data acquired from an image to be sensed 
is color data representing a skin color, filters 
corresponding to those three pieces of band information 
shown in Fig. 6, which are designated by the color data 
are selected from a plurality of filters of the 
multi-spectrum camera. By sensing the image by using 
each filter and acquiring these three pieces of band 
information, multi spectral distribution data 
corresponding to the color data of a skin color can be 
estimated. 

One method of estimating multi spectral 
distribution data R(/l) corresponding to the color data 
of a skin color is to use a linear sum. 

In this method, the above-mentioned three pieces 
of band information for the multi spectral distribution 
data R(/L) corresponding to the color data of a skin 
color are defined as first band information LI ( X ) , 
second band information L2 ( X ) , and third band 



information L3 ( X ) , which are functions concerning the 
wavelength X . The multi spectral distribution data 
R(X) corresponding to the color data of a skin color 
can be estimated by 

R(X) = SaUl-LlCl) + a2-L2(^) + a3-L3(A)} 

■ . . (2) 

where al, a2, and a3 are arbitrary coefficients which 
can be different from one wavelength X to another 

Compared to the band information different in 
wavelength required to estimate the multi spectral 
distribution data of a bluish color, the wavelength 
regions of band information sometimes overlap as in the 
case of the three pieces of band information for 
estimating the multi spectral distribution data 
corresponding to a skin color. These three pieces of 
band information for estimating the multi spectral 
distribution data corresponding to a skin color are 
also three main component spectral distribution data 
for estimating the multi spectral distribution data 
corresponding to a skin color. 

Similar to the estimation of the multi spectral 
distribution data of a bluish color by the 
interpolation method or the linear sum method, this 
main component spectral distribution data can be 
obtained by combining arbitrary band information 
differing in wavelength. 

Accordingly, to accurately sense a specific color 



of high frequency, such as a skin color, it is better 
not to estimate the multi spectral distribution data 
from a large number of pieces of band information 
different in wavelength by using the interpolation 
method. That is, it is preferable to perform image 
sensing by the multi-spectrum camera by using a 
dedicated filter corresponding to a specific color as 
described above, acquire the main component spectral 
distribution data, and then estimate the multi spectral 
distribution data. In this way, it is possible to 
reduce the number of filters for use in image sensing 
and obtain good multi spectral distribution data 
estimation results . 

In the arrangement of the multi-spectrum camera 
of the image sensing device 101 according to this 
embodiment, filter characteristics for use in 
multi-spectrum camera image sensing have a plurality of 
band characteristics different in wavelength. The 
camera is also equipped with dedicated filters for 
previously defined specific colors. By giving band 
information to these dedicated filters, it is possible 
to equally handle the filters having band 
characteristics different in wavelength and these 
dedicated filters for previously defined specific 
colors . 

Accordingly, in the selection of filters for 
multi-spectrum camera image sensing by the image 



sensing device 101 or in the selection of band 
information when image data is stored from the image 
sensing device 101 into the data storage unit 105, 
necessary band information are made to correspond to 
color data beforehand, and band information designated 
by acquired color data is selected. 
[Operation of Image Sensing Device] 

On the rotary filter of the multi-spectrum camera 
sensing unit 103, filters corresponding to band 
information necessary to estimate the multi spectral 
distribution data of color data are arranged in 
accordance with band numbers. Also, for color data 
(for a color designated by color data) sensed by the 
color data sensing unit 102, band information necessary 
to estimate the multi spectral distribution data are 
defined beforehand. By designating a band number, the 
defined band information can be acquired by using a 
filter corresponding to the band number. 

That is, a look-up table as shown in Fig. 7 
defines the band numbers and the number of band 
information necessary to estimate the multi spectral 
distribution data of color data in the image sensing 
device 101. 

Also, a look-up table as shown in Fig. 8 defines 
those filter numbers of the multi-spectrum camera 
sensing unit 103, which correspond to the band numbers 
in the image sensing device 101, and the spectral 



distribution characteristics of these filters. 

Fig. 9 is a block diagram showing detailed 
functions of the image sensing device 101. Fig. 10 is 
a flow chart showing the image sensing procedure of the 
image sensing unit 101. Note that the look-up tables 
shown in Figs. 7 and 8 are stored in a band count value 
storage 602 and a band number storage 603, respectively. 

In step S801, the color data sensing unit 102 
acquires the color data of each pixel to be sensed, and 
the acquired color data of each pixel is stored in a 
data storage unit 607. 

In step S802, the number of band information 
necessary to estimate the multi spectral distribution 
data is acquired as a count value n from the band count 
value storage 602 by referring to the acquired color 
data, and is set in a counter unit 604. Subsequently, 
in step S803 a band number corresponding to a count 
value n-1 is acquired from the band number storage 603. 
In step S804, the acquired band number is supplied to a 
filter selector 605 to select a filter corresponding to 
the band number. Consequently, the rotary filter 
rotates to set the selected filter in front of the CCD 
sensor of the multi-spectrum camera sensing unit 103. 

In step S805, the multi-spectrum camera sensing 
unit 103 senses a spectrum. In step S806, the counter 
unit 604 decrements the count value n. In step S807, 
whether count value n > 0 is checked, and the 



processing from step S803 to step S806 is repeated the 
same number of times as the number of the band 
information. 

After spectrum sensing is performed the same 
number of times as the number of band information, the 
flow advances to step S808, and the band information 
(spectral distribution data) equal in number to the 
bands corresponding to the acquired color data are 
stored as image data in the data storage unit 607. 

The process shown in the flow chart of Fig. 10 is 
repetitively performed for all pixels. The acquired 
color data and the band information equal in number to 
the bands corresponding to the color data are output to 
the image processing apparatus 104. 

The image sensing device 101 can also perform the 
following processing. That is, in the steps after the 
acquisition of the color data in step S801, all filters 
of the multi-spectrum camera are used to acquire the 
spectral distribution data of all pixels. When this 
data is stored from the image sensing device 101 into 
the data storage unit 105, only that spectral 
distribution data of each pixel of an input image, 
which is defined by the acquired color data is stored. 
After that, image processing using the spectral 
distribution data of each pixel stored in the data 
storage unit 105 is similarly performed. 
[Operation of Image Processing Apparatus] 



The processing of the image processing apparatus 
by which the multi spectral distribution data of each 
pixel is estimated from the image data acquired by the 
image sensing unit 101 and output to the output device 
112 will be described next. 

Fig. 11 is a block diagram showing detailed 
functions of the image processing apparatus 104. 
Fig. 12 is a flow chart showing the process procedure 
of this image processing apparatus 104. Note that of 
the input image data from the image sensing device 101, 
the color data of each pixel is stored in a color data 
storage 701 of the data storage unit 105, and the band 
information of each pixel is stored in a spectral 
distribution data characteristic output value storage 
706 of the data storage unit 105. 

Note also that look-up tables as shown in Figs. 7 
and 8 are stored in a band count value storage 702 and 
a band number storage 703, respectively, of the color 
data look-up unit 107. These look-up tables can also 
be acquired from the band count value storage 602 and 
the band number storage 603 of the image sensing device 
101 and stored in the color data look-up unit 107. 

In step S901, the multi spectral distribution 
data estimation processor 106 reads out the color data 
of a pixel of interest from the color data storage 701. 
In step S902, the multi spectral distribution data 
estimation processor 106 refers to the color data and 



acquires from the band count value storage 702 a count 
value n as the number of band information required to 
estimate the multi spectral distribution data. In step 
S903, the multi spectral distribution data estimation 
processor 106 acquires from the band number storage 703 
a band number corresponding to the count value. 

On the basis of the acquired band number, the 
multi spectral distribution data estimation processor 
106 acquires the band spectral distribution data 
characteristic from a spectral distribution data 
characteristic storage 705 in step S904, and reads out 
the band information from the spectral distribution 
data characteristic output value storage 7 06 in step 
S905. 

In step S906, a counter unit 704 decrements the 
count value n. In step S907, whether count value n > 0 
is checked, and the processing from step S903 to step 
S905 is repeated the same number of times as the number 
of the band information. 

Although not shown in the table shown in Fig. 1, 
spectral distribution data configuration information or 
the like for designating the generation of an arbitrary 
number of main component spectral distribution data for 
estimating the multi spectral distribution data from 
the "band spectral distribution data characteristic and 
band information" with respect to the color data is 
defined beforehand. This spectral distribution data 



configuration information includes main component 
spectral distribution data generation designation 
information, the number of main component spectral 
distribution data, band configuration information 
necessary to generate each main component spectral 
distribution data, coefficient information necessary to 
generate each main component spectral distribution data, 
and each main component spectral distribution data for 
estimating the multi spectral distribution data 
corresponding to the color data. 

If in step S908 it is determined by referring to 
the color data that the generation of the main 
component spectral distribution data is not designated, 
the band spectral distribution data characteristics 
acquired in step S904 and the band information acquired 
in step S905 are supplied to the multi spectral 
distribution data estimator 109. 

On the other hand, if in step S908 it is 
determined by referring to the color data that the 
generation of the main component spectral distribution 
data is designated, the band spectral distribution data 
characteristics acquired in step S904 and the band 
information acquired in step S905 are supplied to the 
main component spectral distribution data generator 108. 

In step S909, on the basis of the "band spectral 
distribution data characteristics and band information" 
defined by the color data, the main component spectral 



distribution data generator 108 generates the 
designated main component spectral distribution data 
for estimating the multi spectral distribution data, by 
referring to the number of main component spectral 
distribution data, band configuration information 
necessary to generate each main component spectral 
distribution data, and coefficient information 
necessary to generate each main component spectral 
distribution data. Note that the main component 
spectral distribution data can be obtained by, e.g., 
interpolating the band spectral distribution data 
characteristics configuring the selected main component 
spectral distribution data defined by the color data. 

By using the "band spectral distribution data 
characteristics and band information" equal in number 
to the band information acquired in steps S904 and S905, 
or by using the spectral distribution data for 
estimating the multi spectral distribution data 
corresponding to the color data including the main 
component spectral distribution data generated in step 
S909, in step S910 the multi spectral distribution data 
estimator 109 estimates the multi spectral distribution 
data of a pixel of interest on the basis of the 
spectral distribution data for estimating the multi 
spectral distribution data corresponding to the color 
data including the generated main component spectral 
distribution data, or on the basis of the "band 
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spectral distribution data characteristics and band 
information". In step S911, the estimated multi 
spectral distribution data is stored in the multi 
spectral distribution datastorage 709. Note that the 
multi spectral distribution data can be estimated by 
using each main component spectral distribution data 
and/or the linear sum method using the band spectral 
distribution data characteristics and band information. 

The processing indicated by the flow chart in 
Fig. 12 is repeatedly performed for all pixels to 
calculate the multi spectral distribution data of each 
pixel of the input image. This multi spectral 
distribution data is stored in a multi spectral 
distribution data storage 709. 

By using that multi spectral distribution data of 
each pixel of the input image, which is estimated by 
the above processing, the image processing apparatus 
104 causes the color converter 110 to perform color 
conversion for the multi spectral distribution data to 
obtain a desired output result, thereby obtaining the 
converted multi spectral distribution data. In 
addition, the color data converter 111 integrates the 
multi spectral distribution data converted by the color 
converter 110, by a convolution operation using a color 
matching function of an appropriately selected 
colorimetric system, thereby converting into color 
vector data having three values. As this color vector 



data having three values, color space data unique to a 
device such as the L*a*b* colorimetric system or the XYZ 
colorimetric system is generally selected. Fig. 16 
shows a color matching function in the XYZ colorimetric 
system, as an example of a color matching function. 

The obtained color vector data having three 
values is transmitted from the image processing 
apparatus 104 to the output device 112. The output 
device 112 converts the color vector data having three 
values into output signal data, which is optimum in an 
observation environment, of this output device 112. 

Examples of the output device 112 are a display 
using an RGB signal values, a printer using CMYK signal 
values, and a display and printer using multicolor 
signal values other than RGB and CMYK. This output 
device has a function, such as a color management 
system (CMS) , for obtaining an appropriate output, by 
taking account of the output characteristics of the 
output device, with respect to the three-valued color 
data. Therefore, the color data converter 111 uses 
this CMS to generate an output signal appropriate for 
the output device 112. 

By the above processing, it is possible to set 
the bands and the number of the bands, corresponding to 
a color specified by the color data of an input image, 
of spectral distribution data to be sensed by the 
multi-spectrum camera, and to estimate the spectral 



distribution data from this band information. In 
addition, the spectral distribution data is estimated 
by changing the band information and the number of the 
band information in accordance with the color data 
value, thereby realizing image signal processing using 
multi spectral distribution data. 
[Compression Process] 

When the color data and the band information are 
transmitted from the image sensing device 101 to the 
image processing apparatus 104, if a data compressor 
for compressing the band information is used to 
compress the data amount of the band information (image 
data) as small as possible and transfer the compressed 
data, the time of the data transfer can be shortened. 
A data expander for expanding the compressed band 
information is of course prepared in the image 
processing apparatus 104 to restore the compressed band 
information and store the restored band information in 
the data storage unit 105. 

The process of compressing image data having 
multi spectral distribution data or compressing band 
information will be explained below. 

Fig. 13 is a view showing the relationship 
between multi spectral distribution data and frame band 
information. To acquire the band information of each 
pixel, one frame band information is acquired for one 
wavelength. 



Noting the band information of a pixel of 
interest in each frame, differences between adjacent 
pieces of frame band information are obtained, and 
correlations (Fig. 14) between these pieces of frame 
band information are extracted. Next, differences 
between pixels are obtained in the band information of 
one frame, and correlations between these pixels in the 
one-frame band information are extracted. On the basis 
of the correlations between the pieces of frame band 
information and the correlations between the pixels in 
the one-frame band information, optimum encoding is 
performed. Consequently, spectral distribution data 
can be efficiently compressed. 

Compared to the compression of the spectral 
distribution data described above, the compression of 
the spectral distribution data characteristics does not 
always require frame band information but need only use 
data of each wavelength. Therefore, differences 
between data of different wavelengths of the spectral 
distribution data characteristics are obtained and 
properly encoded. In this manner, the spectral 
distribution data characteristics can be efficiently 
compressed. 

In this embodiment as described above, the multi 
spectral distribution data can be obtained by acquiring 
the spectral distribution band information by using a 
filter which is designated by the acquired color data 



of each pixel and has an arbitrary number of spectral 
distribution characteristics, without using all filters 
mounted on the multi-spectrum camera. Accordingly, it 
is possible to minimize the output data amount from the 
multi-spectrum camera and the load (time) of image 
sensing. 

In addition, in the image processing which 
outputs image data constructed by an input color data 
value and spectral distribution data, the configuration 
of the spectral distribution data is defined in 
accordance with the color data value. This makes it 
possible to estimate the spectral distribution data of 
an object by using a minimum spectral distribution band 
information configuration. Therefore, the processing 
load (processing time) of the image processing can be 
reduced. 

Furthermore, in image data constructed by an 
input color data value and spectral distribution data, 
the configuration of the spectral distribution data is 
defined in accordance with the color data value. 
Therefore, the spectral distribution data can be 
estimated by using minimum spectral distribution band 
information corresponding to the color data. This 
makes it possible to reduce the processing load 
(processing time) of the estimation and provide an 
image processing apparatus which accurately reproduces 
the color of object. 



Modifications 

The spectral distribution data as output band 
information from the multi-spectrum camera includes 
output values of a plurality of wavelength data output 
values. In this embodiment, of an arbitrary number of 
wavelength data in the spectral distribution data, the 
spectral characteristic data of color data need only be 
obtained for at least three wavelength data. For 
example, three wavelength data of a plurality of 
arbitrary wavelength data in the spectral distribution 
data can be RGB spectral characteristic data shown in 
Fig. 15. Instead of this RGB spectral characteristic 
data, a data configuration directly having RGB values 
as color data can also be used. 

In the above embodiment, the acquisition of color 
data by a discrete sensor is explained as an example. 
However, it is also possible to obtain an output value 
having the spectral characteristic data of color data 
by the multi-spectrum camera, calculate the color data 
from the obtained output value, and perform the above 
processing for this calculated color data. 

This embodiment is not restricted to the use of 
the multi-spectrum camera using the rotary filter but 
can also be a multi-spectrum camera comprising an area 
sensor in which filters sensitive to a plurality of 
wavelengths are arranged. Furthermore, the embodiment 
can also be a multi-spectrum camera which switches a 
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plurality of light sources each for emitting light 
having a specific wavelength and which comprises a line 
sensor in which filters sensitive to a plurality of 
wavelengths are switched. That is, the embodiment need 
only be a multi-spectrum camera capable of acquiring 
band information having predetermined spectral 
distribution data characteristics, regardless of the 
optical arrangement of the camera. 

As many apparently widely different embodiments 
of the present invention can be made without departing 
from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the 
specific embodiments thereof except as defined in the 
appended claims. 



